Processing Audio with an Audio Processing Operation

ABSTRACT

An apparatus or method to allow a user to control an audio processing operation of an internal and/or external microphone(s). The method includes providing a configurable user interface which enables mixing of audio signals through interaction with the user interface. The audio signals may be captured with internal or external micorphones.

FIELD OF THE APPLICATION

The present invention relates to apparatus for processing of audiosignals. The invention further relates to, but is not limited to,apparatus for processing audio and speech signals in portable devices.

SUMMARY OF THE APPLICATION

In telecommunications apparatus, a microphone or microphone array istypically used to capture the acoustic waves and output them aselectronic signals representing audio or speech which then may beprocessed and transmitted to other devices or stored for later playback.Currently technologies permit the use of more than one microphone withina microphone array to capture the acoustic waves, and the resultantaudio signal from each of the microphones may be passed to an audioprocessor to assist in Isolating a wanted acoustic wave.

Furthermore video and audio recording (or capture) in electronic devicesother than dedicated cameras is becoming popular. As image recordingquality progressively increases on such electronic devices, they areused for more than ‘amateur’ recording of events such as music concerts,family events, interviews etc. which would have previously required theuse of dedicated audio and video recording apparatus, for example thefilm “The Commuter” was shot on a Nokia N8.

Typical video recording capability on mobile apparatus enables a user toadjust the image quality or change the camera quickly so that a user mayzoom in or out (using either a digital or optical or a combination ofdigital and optical zooming technology) or may change other recordingparameters such as flash, image brightness or contrast, etc. The resultof changing of any of these parameters can be clearly seen by the userin such implementations and as such poor quality video capture can bequickly caught and the parameters adjusted to produce an improvedrecording. However, audio recording capability has not followed theseimprovements.

Typically the user or operator of audio recording apparatus is nottechnically aware of the sound properties being recorded and thus maynot be aware of the sound levels or in which direction the sound iscoming from and thus may not catch when a poor or inaccurate audiorecording is in progress and therefore may be unable to select or adjustthe recording capability of the device to improve the recording.Furthermore even when apparatus has been designed to provide someassistance to the user, it often is displayed in a form which the useris unable to interact with.

Furthermore the audio recording apparatus/video recording apparatus istypically built with the microphone or microphone array integral withthe device. Where the sound source is at some distance and where thecapture or recording is occurring in an environment which is noisy thenthe audio can be muffled or completely lost.

In some circumstances moving the microphone (for example a microphonelocated within the device) to attempt to improve the audio quality canbe problematic for the recording the video or image of the source.Although in some circumstances a recording can be improved by usingexternal directional microphones to provide a directional recordingbeam, an inexperienced operator could select a poor quality externalmicrophone or captured audio signal over the captured audio signalproduced from the electronic device due to lack of or poor qualityinformation. Furthermore control of external microphone/audio captureapparatus is typically limited and is difficult to integrate withintegral apparatus microphone/audio capture apparatus.

This invention proceeds from the consideration that information aboutexternal microphones or microphones external to the apparatus can assistthe apparatus in the control of audio recording and thus, for exampleassist in the reduction of noise of the captured audio signals by audiomicrophone selection and balancing.

SUMMARY OF VARIOUS EXAMPLES

Embodiments of the present invention aim to address the above problem.

There is provided according to a first aspect of the application thereis provided a method comprising: determining at least one audio source;generating a visual representation associated with the at least oneaudio source; and processing an audio signal associated with the atleast one audio source dependent on interaction with the visualrepresentation.

The visual representation may comprise at least one of: a visualrepresentation of the at least one audio source; a visual representationof the type of audio source; a visual representation of the couplingbetween the audio source and an apparatus; a visual representation of atleast one parameter associated with the audio source.

The visual representation of the type of audio source may comprise atleast one of: a visual representation of a microphone integral with anapparatus; a visual representation of a microphone physically separateto an apparatus; a visual representation of a microphone array; a visualrepresentation of a broadcast audio source; a visual representation of aFM broadcast audio source; a visual representation of a T-coil broadcastaudio source; and a visual representation of a near field communication(NFC) audio source.

The visual representation of the coupling between the audio source andan apparatus may comprise at least one of: a visual representation of aphysical coupling between the audio source and the apparatus; a visualrepresentation of a wireless coupling between the audio source and theapparatus; a visual representation of a bluetooth coupling between theaudio source and the apparatus; a visual representation of a NFCcoupling between the audio source and the apparatus; and a visualrepresentation of a T-coil coupling between the audio source and theapparatus.

The visual representation of at least one parameter associated with theaudio source may comprise at least one of: a visual representation of anaudio source status; a visual representation of an audio source gain; avisual representation of an audio source filtering; and a visualrepresentation of mixing between at least two audio sources.

The method may further comprise: sensing a position of the at least oneaudio source; and displaying the visual representation dependent on theposition of the at least one audio source.

The method may further comprise interacting with the visualrepresentation of the audio source.

Interacting with the visual representation of the audio source maycomprise at least one of: selecting the visual representation; andmoving the visual representation on the display.

Processing an audio signal associated with the at least one audio sourcemay comprise at least one of: switching the audio signal; filtering theaudio signal; and amplifying the audio signal.

Filtering the audio signal may comprise at least one of: frequencyfiltering the audio signal; time filtering the audio signal; and spatialfiltering the audio signal.

According to a second aspect there is provided an apparatus comprisingat least one processor and at least one memory including computerprogram code the at least one memory and the computer program codeconfigured to, with the at least one processor, cause the apparatus atleast to perform: determining at least one audio source; generating avisual representation associated with the at least one audio source; andprocessing an audio signal associated with the at least one audio sourcedependent on interaction with the visual representation.

Generating the visual representation may cause the apparatus to performat least one of: a visual representation of the at least one audiosource; a visual representation of the type of audio source; a visualrepresentation of the coupling between the audio source and anapparatus; and a visual representation of at least one parameterassociated with the audio source.

The visual representation of the type of audio source may comprise atleast one of: a visual representation of a microphone integral with anapparatus; a visual representation of a microphone physically separateto an apparatus; a visual representation of a microphone array; a visualrepresentation of a broadcast audio source; a visual representation of aFM broadcast audio source; a visual representation of a T-coil broadcastaudio source; and a visual representation of a near field communication(NFC) audio source.

The visual representation of the coupling between the audio source andan apparatus may comprise at least one of: a visual representation of aphysical coupling between the audio source and the apparatus; a visualrepresentation of a wireless coupling between the audio source and theapparatus; a visual representation of a bluetooth coupling between theaudio source and the apparatus; a visual representation of a NFCcoupling between the audio source and the apparatus; and a visualrepresentation of a T-coil coupling between the audio source and theapparatus.

The visual representation of at least one parameter associated with theaudio source may comprise at least one of: a visual representation of anaudio source status; a visual representation of an audio source gain; avisual representation of an audio source filtering; and a visualrepresentation of mixing between at least two audio sources.

The apparatus may further perform: sensing a position of the at leastone audio source; and displaying the visual representation dependent onthe position of the at least one audio source.

The apparatus may further perform interacting with the visualrepresentation of the audio source.

Interacting with the visual representation of the audio source mayfurther cause the apparatus to perform at least one of: selecting thevisual representation; and moving the visual representation on thedisplay.

Processing an audio signal associated with the at least one audio sourcemay cause the apparatus to perform at least one of: switching the audiosignal; filtering the audio signal; and amplifying the audio signal.

Filtering the audio signal may further cause the apparatus to perform atleast one of: frequency filtering the audio signal; time filtering theaudio signal; and spatial filtering the audio signal.

According to a third aspect of the application there is provided anapparatus comprising: an audio source determiner configured to determineat least one audio source; a visualizer configured to generate a visualrepresentation associated with the at least one audio source; and acontroller configured to process an audio signal associated with the atleast one audio source dependent on interaction with the visualrepresentation.

The visualizer may be configured to generate the visual representationcomprising at least one of: a visual representation of the at least oneaudio source; a visual representation of the type of audio source; avisual representation of the coupling between the audio source and anapparatus; a visual representation of at least one parameter associatedwith the audio source.

The visual representation of the type of audio source may comprise atleast one of: a visual representation of a microphone integral with anapparatus; a visual representation of a microphone physically separateto an apparatus; a visual representation of a microphone array; a visualrepresentation of a broadcast audio source; a visual representation of aFM broadcast audio source; a visual representation of a T-coil broadcastaudio source; and a visual representation of a near field communication(NFC) audio source.

The visual representation of the coupling between the audio source andan apparatus may comprise at least one of: a visual representation of aphysical coupling between the audio source and the apparatus; a visualrepresentation of a wireless coupling between the audio source and theapparatus; a visual representation of a bluetooth coupling between theaudio source and the apparatus; a visual representation of a NFCcoupling between the audio source and the apparatus; and a visualrepresentation of a T-coil coupling between the audio source and theapparatus.

The visual representation of at least one parameter associated with theaudio source may comprise at least one of: a visual representation of anaudio source status; a visual representation of an audio source gain; avisual representation of an audio source filtering; and a visualrepresentation of mixing between at least two audio sources.

The apparatus may further comprise: at least one sensor configured tolocate the at least one audio source; and a display configured todisplay the visual representation dependent on the position of the atleast one audio source.

The apparatus may further comprise a user interface input configured tointeract with the visual representation of the audio source.

The user interface input configured to interact with the visualrepresentation of the audio source may comprise at least one of: aselector input determiner configured to determine a selection of thevisual representation; and a motion input determiner configured todetermine a moving of the visual representation.

The user interface input may comprise at least one of: a touch screeninterface; a keypad; a keyboard; a touchpad; and a mouse.

The controller may comprise at least one of: a switch configured toswitch the audio signal; a filter configured to filter the audio signal;and an amplifier configured to amplify the audio signal.

The filter may comprise at least one of: a frequency domain filter; atime domain filter; and a spatial domain filter.

According to a fourth aspect of the application there is providedapparatus comprising: means for determining at least one audio source;means for generating a visual representation associated with the atleast one audio source; and means for processing an audio signalassociated with the at least one audio source dependent on interactionwith the visual representation.

The visual representation may comprise at least one of: a visualrepresentation of the at least one audio source; a visual representationof the type of audio source; a visual representation of the couplingbetween the audio source and an apparatus; a visual representation of atleast one parameter associated with the audio source.

The visual representation of the type of audio source may comprise atleast one of: a visual representation of a microphone integral with anapparatus; a visual representation of a microphone physically separateto an apparatus; a visual representation of a microphone array; a visualrepresentation of a broadcast audio source; a visual representation of aFM broadcast audio source; a visual representation of a T-coil broadcastaudio source; and a visual representation of a near field communication(NFC) audio source.

The visual representation of the coupling between the audio source andan apparatus may comprise at least one of: a visual representation of aphysical coupling between the audio source and the apparatus; a visualrepresentation of a wireless coupling between the audio source and theapparatus; a visual representation of a bluetooth coupling between theaudio source and the apparatus; a visual representation of a NFCcoupling between the audio source and the apparatus; and a visualrepresentation of a T-coil coupling between the audio source and theapparatus.

The visual representation of at least one parameter associated with theaudio source may comprise at least one of: a visual representation of anaudio source status; a visual representation of an audio source gain; avisual representation of an audio source filtering; and a visualrepresentation of mixing between at least two audio sources.

The apparatus may further comprise: means for sensing a position of theat least one audio source; and means for displaying the visualrepresentation dependent on the position of the at least one audiosource.

The apparatus may further comprise means for interacting with the visualrepresentation of the audio source.

The means for interacting with the visual representation of the audiosource may comprise at least one of: means for selecting the visualrepresentation; and means for moving the visual representation on thedisplay.

The means for processing an audio signal associated with the at leastone audio source may comprise at least one of: means for switching theaudio signal; means for filtering the audio signal; and means foramplifying the audio signal.

The means for filtering the audio signal may comprise at least one of:means for frequency filtering the audio signal; means for time filteringthe audio signal; and means for spatial filtering the audio signal.

An electronic device may comprise apparatus as described above.

A chipset may comprise apparatus as described above.

BRIEF DESCRIPTION OF DRAWINGS

For better understanding of the present invention, reference will now bemade by way of example to the accompanying drawings in which:

FIG. 1 shows schematically an apparatus employing embodiments of theapplication;

FIG. 2 shows schematically the apparatus shown in FIG. 1 in furtherdetail;

FIG. 3 shows schematically the parameter visualizer shown in FIG. 2 infurther detail;

FIG. 4 shows schematically the apparatus being used for interviewingaccording to some embodiments;

FIG. 5a shows schematically the apparatus being in a studio interview ina first arrangement according to some embodiments;

FIG. 5b shows schematically the apparatus being used in a studiointerview in a second arrangement according to some embodiments;

FIG. 6 shows an example of a switch user interface configurationaccording to some embodiments;

FIG. 7a shows an example of an audio sub-menu user interface indicatoraccording to some embodiments;

FIG. 7b shows an example of an audio sub-menu user interfaceconfiguration showing mixing between internal microphone and externalmicrophone inputs; FIGS. 7c and 7d show examples of an audio sub-menuuser interface configuration showing mixing between three microphoneinputs;

FIG. 8a shows an example of an audio control user interface indicator;

FIGS. 8b and 8c show examples of an audio control user interfaceconfiguration for controlling microphone inputs;

FIG. 8d shows an example of an audio control user interfaceconfiguration showing a slider mixing control between an external andinternal microphone input;

FIGS. 8e, and 8f show examples of an audio control user interfaceconfiguration showing a two slider mixing control controlling theselection of an external and internal microphone input;

FIG. 8g shows an example of an audio control user interfaceconfiguration showing a two slider mixing control with displayed inputsignal history;

FIGS. 8h, 8i, and 8j show examples of audio control user interfaceconfiguration controlling the mixing of three audio inputs;

FIG. 9 show an augmented reality view configuration suitable inembodiments of the application; and

FIG. 10 shows an overview of the operation of embodiments of theapplication.

EMBODIMENTS OF THE APPLICATION

The following describes apparatus and methods for the provision ofenhancing video and/or audio capture and recording flexibility usingexternal microphones. In this regard reference is first made to FIG. 1which shows a schematic block diagram of an exemplary electronic device10 or apparatus, which may incorporate enhanced signal captureperformance components and methods.

The apparatus 10 can for example be a mobile terminal or user equipmentfor a wireless communication system. In some other embodiments theapparatus can be any audio player, such as an mp3 player or mediaplayer, equipped with suitable microphone and sensors as describedbelow.

The apparatus 10 in some embodiments comprises a processor 21. Theprocessor 21 may be configured to execute various program codes. Theimplemented program codes may comprise audio capture/recordingenhancement code.

The implemented program codes can in some embodiments be stored forexample in the memory 22 for retrieval by the processor 21 wheneverneeded. The memory 22 in some embodiments can provide a program codesection 23 for the storage of the program codes. The memory 22 in somefurther embodiments could further provide a data storage section 24 forstoring data, for example data that has been processed in accordancewith the embodiments.

The audio capture/recording enhancement code may in embodiments beimplemented at least partially in hardware or firmware.

The processor 21 can in some embodiments be linked via adigital-to-analogue converter (DAC) 32 to a playback speaker 33.

The digital to analogue converter (DAC) 32 can be any suitableconverter.

The playback speaker 33 can, for example, be any suitable audiotransducer equipment suitable for producing acoustic waves for theuser's ears generated from the electronic audio signal output from theDAC 32. The playback speaker 33 in some embodiments can be a headset orplayback speaker and can be connected to the electronic device 10 via aheadphone connector. In some embodiments the speaker 33 can comprise theDAC 32. Furthermore in some embodiments the speaker 33 can connect tothe apparatus 10 wirelessly, for example by using a low power radiofrequency connection such as demonstrated by the Bluetooth A2DP profile.As described herein the connection implemented by the wirelessconnection may be managed by the transceiver 13 in some embodiments ofthe application.

The processor 21 in some embodiments can be further linked to atransceiver (TX/RX) 13, to a user interface (UI) 15 and to a memory 22.

The user interface 15 can in some embodiments enable a user to inputcommands to the apparatus 10, for example via a keypad, and/or to obtaininformation from the apparatus 10, for example via a display (notshown). It would be understood that the user interface can furthermorein some embodiments be any suitable combination of input and displaytechnology, for example a touch screen display suitable for bothreceiving inputs from the user and displaying information to the user.

The transceiver 13, can be any suitable communication technology and beconfigured to enable communication with other electronic devices, forexample via a wireless communication network.

The apparatus 10 can in some embodiments further comprise at least oneintegral microphone or microphone array 11 for inputting or capturingacoustic waves and outputting audio or speech signals to be processedaccording to embodiments of the application. The audio or speech signalscan according to some embodiments be transmitted to other electronicdevices via the transceiver 13 or can be stored in the data section 24of the memory 22 for later processing.

A corresponding program code or hardware to control the capture of audiosignals using the at least two microphones can be activated to this endby the user via the user interface 15.

The apparatus 10 in some embodiments can further comprise ananalogue-to-digital converter (ADC) 14 configured to convert the inputanalogue audio signals from the microphone 11 into digital audio signalsand provide the digital audio signals to the processor 21.

The apparatus 10 may in some embodiments receive the audio signals froma microphone not implemented physically on the electronic device. Forexample the speaker 33 apparatus in some embodiments may comprise anexternal microphone array. In some embodiments the apparatus 10 canreceive the audio signals from at least one external microphone, such asexternal microphone 31 shown in FIG. 1. The external microphone 31 canthen in some embodiments transmit the audio signals from the externalmicrophone 31. In some embodiments the external microphone 31 could bean integral microphone implemented in a further apparatus or anotherelectronic device. The external microphone 31 in some embodiments can bewirelessly coupled to the apparatus 10 for example via a suitablewireless protocol, such as for example a Bluetooth channel, a Wi-Fichannel, a magnetic loop channel (such as a T-coil reception), or lowpower FM channel. In some embodiments the external microphone can becoupled to the apparatus via a wired connection, for example via thephysical socket and plug connection. Thus for example an externalmicrophone or external source could be in some embodiments an audiosource broadcasted on a FM frequency (such as the Obama inauguration, ora FM radio station) and received by the apparatus FM receiver which asdescribed herein could be added as an alternative audio input.Furthermore in some embodiments the external source could be in someembodiments an mp3 format audio source being transmitted using a nearfield communication (NFC) format from an ‘intelligent’ billboardadvertising a new single from an artist.

In some embodiments, the processor 21 may execute the audiocapture/recording enhancement program code stored in the memory 22. Theprocessor 21 in these embodiments may process the received audio signaldata, and output or store the processed audio data.

The received audio data can in some embodiments be stored, instead ofbeing processed immediately, in the data section 24 of the memory 22,for instance for later processing and presentation or forwarding tostill another apparatus or electronic device.

Furthermore the apparatus can in some embodiments comprise sensors or asensor bank 16. The sensor bank 16 can in such embodiments receiveinformation about the environment in which the apparatus 10 is operatingand pass this information to the processor 21 in order to affect theprocessing of the audio signal and in particular to affect the processor21 in audio capture/recording applications. The sensor bank 16 can insuch embodiments comprise at least one of the following set of sensors.

The sensor bank 16 can in some embodiments comprise a camera module. Thecamera module can in some embodiments comprise at least one camerahaving a lens for focusing an image on to a digital image capture meanssuch as a charged coupled device (CCD). In some other embodiments thedigital image capture means can be any suitable image capturing devicesuch as complementary metal oxide semiconductor (CMOS) image sensor. Thecamera module can further comprise in some embodiments a flash lamp forilluminating an object before capturing an image of the object. Theflash lamp is in such embodiments linked to a camera processor forcontrolling the operation of the flash lamp. In other embodiments thecamera may be configured to perform infra-red and near infra-red sensingfor low ambient light sensing. The camera can be also linked to thecamera processor for processing signals received from the at least onecamera before passing the processed image to the processor. The cameraprocessor can in some embodiments be linked to a local camera memorywhich can store program codes for the camera processor to execute whencapturing an image. Furthermore the local camera memory can in suchembodiments be used in some embodiments as a buffer for storing thecaptured image before and during local processing. In some embodimentsthe camera processor and the camera memory are implemented within theprocessor 21 and memory 22 respectively.

Furthermore in some embodiments the camera module may be physicallyimplemented separate from the apparatus and coupled to the apparatuseither wirelessly or by physical connection. For example in someembodiments the camera module can be implemented as part of a combinedplayback speaker/external microphone/cameral module apparatus to be wornon the head.

In some embodiments the sensor bank 16 comprises a position/orientationsensor. The orientation sensor in some embodiments can be implemented bya digital compass or solid state compass configured to determine theapparatus orientation with respect to the horizontal axis. In someembodiments the position/orientation sensor can be a gravity sensorconfigured to output the apparatus's orientation with respect to thevertical axis. The gravity sensor for example can be implemented as anarray of mercury switches set at various angles to the vertical with theoutput of the switches indicating the angle of the apparatus withrespect to the vertical axis.

In some other embodiments the position/orientation sensor is anaccelerometer or gyroscope. In other words in some embodiments there cancomprise means for sensing a position of a determined at least one audiosource.

It is to be understood again that the structure of the apparatus 10could be supplemented and varied in many ways.

It would be appreciated that the schematic structures described in FIG.2 and the method steps in FIG. 10 represent only a part of the operationof a complete audio capture/recording chain comprising some embodimentsas exemplarily shown implemented in the apparatus shown in FIG. 1.

With respect to FIG. 2 some embodiments of the application asimplemented and operated are shown in further detail.

Furthermore with respect to FIG. 10, there is a flow chart showing aseries of operations which may be employed in some embodiments of theapplication.

In some embodiments the apparatus provides a user or operator of anapparatus an interactive flexible audio and/or audio visual recordingsolution. The user interface 15 can in these embodiments provide theuser the information required from the recorded audio signals bymeasuring and displaying the sound field in real time so that theoperator or user of the apparatus can determine what is being recorded.Furthermore in some embodiments, using the same user interface theoperator of the apparatus can also adjust parameters in real time andthus adjust the recorded sound field and so avoid recoding or capturingpoor quality audio signals.

The apparatus in some embodiments as described previously comprises atleast one microphone. As shown in FIG. 2 the following examples show aninternal microphone 11 coupled to the apparatus using a physicalcoupling, for example an internal coupling between the microphone 11 andprocessor (via in some embodiments by a suitable analogue to digitalconvertor). It would be understood that in some other embodiments therecan be apparatus with multiple internal microphones or a steerable arrayof microphones which can be controlled by the user using the visualizerdescribed herein.

Furthermore as shown in FIG. 2, these examples show two externalmicrophones, a first external microphone 31 a coupled wirelessly to theapparatus 10 and a second external microphone 31 b also coupledwirelessly to the apparatus. Although two external microphones and oneinternal microphone are shown in this example it would be understoodthat in some embodiments any suitable number of microphones can be‘displayed’ and ‘controlled’, and that the microphones can be wirelesslyor physically coupled to the apparatus in any suitable combination ofmicrophones. For example in some embodiments at least some of theexternal microphones can be implemented on a supporting structure, suchas a NH905 Nokia headset which is equipped with 10 microphones, whichcould be detected, displayed and controlled as described herein andpermit surround sound recording.

In some embodiments the apparatus 10 comprises an audio sourcecontroller 101. The audio source controller 101 can be any suitablemeans for controlling the various audio sources such as the internalmicrophone 11, external microphone 31, and in some embodimentspre-stored audio sources. An audio source can be described as anysuitable source of audio information and can in some embodiments be adevice or apparatus away from the original acoustic sources, for examplean external microphone can in some embodiments be an audio sourcesuitable for capturing the audio waves generated from an acoustic sourcesuch as a subject being interviewed or a choir singing.

In some embodiments the audio source controller 101 can comprise anaudio source determiner 102 configured to determine/search for audiosources. In some embodiments the audio source determiner 102 isconfigured to determine whether there are any external microphones oraudio sources which can be suitably coupled to the apparatus 10.

Furthermore in some embodiments the audio source determiner 102 can beconfigured to inquire or determine the status of each of the detectedexternal microphone or audio sources. For example in some embodimentsthe audio source determiner 102 can be configured to determine anexternal microphone plug having been inserted into a suitable apparatus.In some other embodiments the audio source determiner 102 can beconfigured to determine (or detect) any suitable wireless couplingexternal microphones are in range and initialise a coupling orhandshaking such that the external microphone is configured to be ableto send audio signals to the apparatus. In some further embodiments theaudio source determiner 102 can be configured to operate the transceiver13 such that any ‘in range’ wireless audio source is detected, forexample a FM audio source, a magnetic loop audio source or Wi-Fi audiosource, and then the transceiver 13 is controlled to receive or ‘tunein’ to the audio source.

The operation of searching for audio sources or external microphones isshown in FIG. 10 by step 901.

The audio source controller 101 in some embodiments furthermore canreceive from the sensors 16 further information which can be associatedwith each of the audio sources. For example for an external microphone31 the sensors 16 in some embodiments can be configured to determine thelocation and orientation of the external microphone relative to theapparatus. In other words in at least one embodiment there can comprisea means for determining at least one audio source. Furthermore in suchembodiments for example equipped with multiple or 3D cameras thelocation or position of audio sources could be estimated with respect tothe space or environment surrounding the apparatus.

In some embodiments the audio source controller 101, can receiveinformation associated with each determined (or detected) audio sourcefrom the audio source. For example in some embodiments the externalmicrophone 31 can transmit with the audio signal an estimate of theexternal microphone 31 position and orientation. In such embodiments theaudio source controller 101 can compare the received information to theapparatus information generated by the sensors 16. For example arelative distance and orientation from the apparatus can be estimated insuch a manner. In some other embodiments the external microphone 31 oraudio source can broadcast a beacon signal which can be detected by thesensors and used to produce an estimate of the location of the externalmicrophone. For example the external microphone 31 can be configured tooutput an infra-red beacon detected by the camera and enabling a visuallabel as described herein to be associated with and overlaid on thevisual image.

The information determined with or associated with each microphone canbe any suitable information such as but not exclusively: microphonestatus (on, off, standby); microphone status (receiving audio signals,transmitting/passing audio signals); microphone movable/static;microphone position; microphone orientation, microphone faultdetermination; microphone type; microphone available connectivity; andmicrophone signal strength.

The operation of determining microphone information or parameter valuesfor internal and/or external microphones is shown in FIG. 10 by step903.

The apparatus 10 furthermore can in some embodiments comprise aparameter/signal visualiser 105. The parameter/signal visualiser 105 canin some embodiments receive the parameter settings associated with eachof the determined microphones and further in some embodiments the audiosignals associated with each determined microphone and determine orgenerate representations associated with each of the audio sources in aform suitable for being displayed. The parameter/signal visualiser 105can thus in some embodiments determine or generate representations forthe audio signals associated with the external microphones or internalmicrophones being received or detected at the apparatus. Theserepresentations can for example be representations associated with theaudio source such as for example identifying whether the audio source isinternal or external to the apparatus, further distinguishing betweenthe types of audio source such as but not exclusively a singlemicrophone, multiple microphones or an array of microphones, a FM audiostream, an internally stored audio stream such an mp3 internal audiosignal, a NFC audio stream, and a T-coil audio stream. Furthermore insome embodiments the representations can be associated with the audiosignal such as for example a time domain representation or frequencydomain representation or the audio signal, a representation of thesignal strength (or signal power), a representation of the relativeposition of the microphone/audio source to the apparatus, arepresentation of the microphone/audio source beam orientation, arepresentation of the status of the audio source (whether the microphoneis switched on, off, or standby), a representation of the microphonegain, and a representation of any filtering applied to audio signal.Thus in at least one embodiment there can comprise a means forgenerating a visual representation associated with the at least oneaudio source.

The visual representation can in at least one embodiment and asdescribed herein comprise at least one of: a visual representation ofthe at least one audio source; a visual representation of the type ofaudio source; a visual representation of the coupling between the audiosource and an apparatus; a visual representation of at least oneparameter associated with the audio source. Furthermore as describedherein the visual representation of the type of audio source cancomprise at least one of: a visual representation of a microphoneintegral with an apparatus; a visual representation of a microphonephysically separate to an apparatus; a visual representation of amicrophone array; a visual representation of a broadcast audio source; avisual representation of a FM broadcast audio source; a visualrepresentation of a T-coil broadcast audio source; and a visualrepresentation of a near field communication (NFC) audio source.Similarly in some embodiments as described herein the visualrepresentation of the coupling between the audio source and an apparatusmay comprise at least one of: a visual representation of a physicalcoupling between the audio source and the apparatus; a visualrepresentation of a wireless coupling between the audio source and theapparatus;

a visual representation of a bluetooth coupling between the audio sourceand the apparatus; a visual representation of a NFC coupling between theaudio source and the apparatus; and a visual representation of a T-coilcoupling between the audio source and the apparatus. Also in someembodiments the visual representation of at least one parameterassociated with the audio source may comprise at least one of: a visualrepresentation of an audio source status; a visual representation of anaudio source gain; a visual representation of an audio source filtering;and a visual representation of mixing between at least two audiosources.

The parameter/signal visualizer 105 can be configured to furthermorepass the visual representations to the user interface 15 and inparticular the user interface display 111.

The operation of generating visual representations associated with theinternal and external microphone audio signals is shown in FIG. 10 bystep 905.

With respect to FIG. 3, the parameter/signal visualizer 105 is shown infurther detail. The parameter/signal visualizer 105 can in someembodiments comprise a switch user interface visualizer 251. The switchuser interface visualizer 251 can in some embodiments be configured togenerate a visual representation of an audio source (for example aninternal or external microphone) and so enable a switching operation tobe carried out.

With respect to FIG. 6 an example visual representation for switchinguser interface operations can be shown. The apparatus 10 as shown inFIG. 1 with a user interface 15 in the form of a touch screen display.The switch user interface visualizer 251 can be configured to receiveinformation on which audio sources are available and/or active andsuitable for transmitting and generate on a region 500 of the touchscreen display iconic representation of the determined audio sources.

As shown in FIG. 6 there can be displayed more than one of theserepresentations, however in some embodiments the switching can be atoggling or cycling through available audio sources with the next to beselectable being displayed. The switch user interface visualizer 251 canfor example generate a first representation 501 of a microphone in a boxor window with an arrow pointing towards the microphone to indicate thatby selecting this representation an external microphone can be selected.Similarly the switch user interface visualizer 251 can for examplegenerate a second representation 503 of a mobile phone or electronicdevice in a box or window with an arrow pointing towards the device toindicate that by selecting this representation an integral or internalmicrophone can be selected. It would be understood that in someembodiments switch user interface visualizer 251 can be configured tooutput a visualization which enables a user to distinguish between notonly whether the audio source is internal (mobile phone representation)or external (microphone representation) but the type of the audio source(for example an array of microphones can be represented by an iconshowing multiple microphones, a FM source by a transistor radio icon orthe letters FM or the logo or representation of the FM signal, aninternal mp3 source by the letters mp3, an internally stored source byan icon representing the coding used on the audio signal for example theogg vorbis logo for ogg vorbis encoded audio signals, a networkstreaming audio source by a streaming icon or the logo or representationof the streaming service for example the Nokia Comes with Music iconwhen streaming from ovi music, a T-coil source by the known T-coilhearing aid icon, a NFC source by a loop icon). Furthermore it would beunderstood that the visualisations produced by the switch user interfacevisualizer 251 could be used in some embodiments by other visualizercomponents to assist the user in determining which source or control isto be selected and/or operated.

The user interface display 111 can be any suitable display technologyconfigured to receive the visual representations from theparameter/signal visualizer 105 and display the representation to theuser. An example of a suitable display technology could be but notexclusively one of LCD, LED, OLED. Furthermore in some embodiments theuser interface 15 comprises an user interface input 113 suitable forreceiving selections and other user input operations. In the hereinexamples the user interface input 113 is a touch screen input associatedwith the user interface display, however it would be appreciated thatthe user interface input 113 could be any one or combination of keypad,keyboard, joystick, trackpad, and in some embodiments be separate fromthe display.

The user interface input 113 can thus in some embodiments receive aninput from the user, for example the user can select the displayedrepresentation of the switch user interface first representation 501.The input can be detected and passed to the user interface inputprocessor 107.

In some embodiments the apparatus comprises an user interface inputprocessor 107. The user interface input processor 107 can be configuredto receive inputs from the user interface input, process these inputsignals and generate control signals to be passed to the audio sourcecontroller 101. In the switching user interface example, the userinterface input processor 107 receives from the user interface input 113an indication that the user has selected the first representation 501associated with the external microphone audio source. The user interfaceinput processor 107 can then in such embodiments generate a switchcontrol signal indicating the external microphone and pass the signal tothe audio source controller 101.

The determination of a control input from the user interface is shown inFIG. 10 by step 907.

In other words in some embodiments the apparatus may further comprisemeans for interacting with the visual representation of the audiosource, where the means for interacting with the visual representationof the audio source may comprise at least one of: means for selectingthe visual representation; and means for moving the visualrepresentation on the display. For example in some embodiments the usercould manually move the estimated position of the external microphone soit better matches the accurate position. Furthermore in some embodimentsalthough a 2D display is described it would be understood that the sameapparatus and methods could be applied to the determination,visualization in 3D and control of audio sources.

In some embodiments the audio source controller 101 comprises controlmodules configured to receive control signals from the user interfaceinput processor 107 and process the detected audio sources dependent onthese control signals. Thus for example in some embodiments the audiosource controller comprises a switch control module 104 configured toroute or switch the audio source signals. Thus in the switching userinterface example the switch control module 104 can receive a controlinput from the user interface input processor 107 and dependent on thecontrol input, which in this example is an indicator to select theexternal microphone audio source, the switch control module 104 isconfigured to route the external microphone audio source to an output(which for this example can be the audio codec 103) for furtherprocessing and/or storage and/or transmission.

In some embodiments the audio source controller 101 comprises a mixermodule 106. The mixer module 106 in a manner similar to the switchmodule can be configured to receive the detected audio source audiosignals and apply a gain and/or mixing to the signals dependent on thecontrol signals received from the UI input processor. Any suitable gainand/or mixing apparatus can be implemented for example controllableanalogue gain, controllable digital gain, time sample mixing, andfrequency domain mixing.

In other words in at least one embodiment there comprises means forprocessing an audio signal associated with the at least one audio sourcedependent on interaction with the visual representation.

Furthermore in some embodiments the audio source controller 101comprises a filter module 108. The filter module 108 can be configuredto receive the detected audio source audio signals and apply a frequencydependent gain to the signals dependent on the control signals receivedfrom the UI input processor 107. Any suitable frequency dependent gainapparatus can be implemented by the filter module 108, for exampleanalogue filters, digital filters, time domain filtering, spatialfiltering (for example for handling microphone arrays) and frequencydomain filtering.

The operation of controlling the audio sources is shown in FIG. 10 bystep 909.

In some embodiments the apparatus 10 comprises an audio codec 103. Theaudio codec 103 can be configured to receive at least one audio signaland encode or decode it into a suitable form. For example the audiocodec 103 can be configured to encode a received audio signal andgenerate a form suitable for storage in memory 22 or passing to aplayback speaker 33. The audio codec 103 can be any suitable codec.

Although herein is described an example of selecting one of the detectedaudio sources for the operation of capturing the audio signal it wouldbe understood that more complex switching operations could beimplemented by using such representations. For example in someembodiments the switch user interface visualizer 251 could be configuredto generate a visualization allowing the selection of the output todirect the source to a specific memory, or to output the source directlyto playback. In other words the means for processing an audio signalassociated with the at least one audio source may comprise at least oneof: means for switching the audio signal; means for filtering the audiosignal; and means for amplifying the audio signal. Furthermore it wouldbe understood that the means for filtering the audio signal may compriseat least one of: means for frequency filtering the audio signal; meansfor time filtering the audio signal; and means for spatial filtering theaudio signal.

In some embodiments the parameter/signal visualizer 105 can in someembodiments comprise an audio sub-menu user interface visualizer 253.The audio sub-menu user interface visualizer 253 can in some embodimentsbe configured to generate a visual representation for enabling controlof audio sources (for example an internal or external microphone).

With respect to FIGS. 7a to 7d examples of visual representationsgenerated by the audio sub-menu user interface visualizer 253 are shown.

The audio sub-menu user interface visualizer 253 can be configured forexample as shown in FIG. 7a to generate a first sub-menu representation600 on the display. The first sub-menu visual representation 600 can bein some embodiments a representation of a microphone in a box or windowto indicate that by selecting this representation a sub-menu can bedisplayed. The first sub-menu visual representation 600 can beconsidered to be a ‘closed’ representation which is configured to ‘open’when selected or interfaced with.

In a manner as described above the first sub-menu visual representation600 can in some embodiments be interfaced with by selecting therepresentation, for example by touching the display within a regiondefined by the representation. The UI input 113 can then in suchembodiments pass the touch selection to the UI input processor 107. Insome embodiments rather than the UI input processor 107 passing thecontrol signal to the audio source controller 101 the UI input processor107 is configured to pass a control signal to the parameter/signalvisualizer 105. In such embodiments the control signal can be used bythe parameter/signal visualizer to change the representation passed tothe display. Thus for example in the sub-menu window example byselecting or touching the sub-menu visual representation 600, the UIinput 113 passes the ‘touch’ input to the UI input processor 107 whichprovides a control signal to the audio sub-menu user interfacevisualizer 253 to open up the sub-menu visual representation (or if opento close the sub-menu visual representation).

With respect to FIG. 7b an example of an ‘open’ audio sub-menu userinterface representation is shown. The ‘open’ audio sub-menu userinterface representation comprises the first sub-menu visualrepresentation 600 and a further ‘open’ or information window 601 withinwhich information and control of the various audio sources can beimplemented. Thus the audio sub-menu user interface visualizer 253 canbe configured in some embodiments to generate the information window 601comprising iconic representations of available audio sources and theirassociated characteristics.

Thus for example in FIG. 7b the audio sub-menu user interface visualizer253 can generate an Iconic representation of the internal microphone (amobile device icon) 603 and associated with the visual representation ofthe internal microphone 603 a visual representation of the gain appliedto the audio signal received from the internal microphone shown by aslider visual representation 605. In some embodiments the user caninteract with either the visual representation of the microphone so thatin some embodiments by ‘touching’ or ‘tapping’ the visual representationor selecting the representation the audio source controller 104 cantoggle switching the microphone audio signal input on and off. In someembodiments by long tapping' the visual representation the UI inputprocessor can be configured to control the visualizer to display furtherinformation on the source such as the type of source, the format of theaudio signal from the source, and the time stamp of the source. In someembodiments by ‘touching’ (for example by moving the slider positionalong the track) the slider visual representation associated with themicrophone or visual representation the UI input processor 107 can beconfigured to control the filter 108 and/or mixer 106 to change the gainapplied to the audio signal. In some further embodiments the slider canbe a 2D representation of the frequency dependent gain applied to thesignal and thus in such embodiments by interacting with the 2Drepresentation and/or changing the shape of the envelope of the slider acontrol signal can be passed to the filter module 108 of the audiosource controller 101 to apply a different frequency dependent gain.

As also shown in FIG. 7b more than one audio source can be displayed inthe further ‘open’ or information window 601. In such embodiments theaudio sub-menu user interface visualizer 253 can generate further iconicrepresentations representing further audio sources, for example ofexternal microphone (a microphone icon) 609. In some embodiments audiosub-menu user interface visualizer 253 can generate visualrepresentations or icons associated with the displayed further audiosources, for example the Bluetooth icon 611 indicating that the externalmicrophone is wireless and connected via a Bluetooth coupling to theapparatus. Other representations reflecting wired, or other wirelesscouplings can be generated in some embodiments. The further iconicrepresentations can further be associated with their associated audiosignal characteristics, such as shown in FIG. 7b by the further slider607.

It would be understood that where the display permits the audio sub-menuuser interface visualizer 253 can generate coloured or shaded visualrepresentations where the colour or shade can indicate furtherparameters or characteristics associated with the audio source. Forexample a detected microphone which is on standby can have a visualrepresentation with a first colour or shading pattern, a microphonewhich is on and sending audio signals can have a visual representationwith a second colour or shading pattern. Other visual tools such astransparency, blinking can also in some embodiments be used to conveyinformation about the audio source.

With respect to FIG. 7c a further example of an example of an ‘open’audio sub-menu user interface representation is shown. The audiosub-menu user interface visualizer 253 in this example is configured togenerate visual representation of the audio sources defined by labelsrather than icons or pictures. Thus for example the internal microphoneis visually represented by the label ‘INTERNAL’ 623 and has anassociated slider 624 showing the associated gain applied to theinternal microphone audio signal. Also generated can be visualrepresentations associated with external microphones such as a firstexternal microphone visually represented by the label ‘EXTERNAL’ 625,indicating an external microphone, the symbol for a Bluetooth couplingand the label ‘1’ indicating it is the first external microphone.Furthermore the external microphone audio source gain can be visuallyrepresented by and controlled by use of a visual representation of afurther slider 626. Furthermore a second external microphone can berepresented by the label ‘EXTERNAL’ 627, indicating an externalmicrophone, the symbol for a Bluetooth coupling and the label ‘2’indicating it is the second external microphone. Furthermore the secondexternal microphone audio source gain can be visually represented by andcontrolled by use of a visual representation of a further slider 628.

Furthermore in some embodiments the parameter/signal visualizer 105 (andthe audio sub-menu user interface visualizer 253) can be configured togenerate visual representations enabling further information or controloptions, such as shown in FIG. 7c by the example option label ‘NOISECANCELATION’ 629 and a selection or tick box 630 indicating the statusof the noise cancellation operation. In such embodiments by selectingeither the label or selection box the option in question can be selectedis currently de-selected or de-selected if currently selected. It wouldbe understood that other effects or filtering to be applied to all ofthe selected audio sources could be controlled using the option labelembodiment similar to that described above. Thus in some embodimentsthere can be a visual representation for a ‘global’ muting where alldetected and/or determined audio sources are muted. In some otherembodiments a noise reduction option can be controlled by theinteraction with a visual representation for a label ‘NOISE REDUCTION’.

Although the above label visual representations are in English it wouldbe understood that any other suitable language or text label can be usedin some embodiments of the application.

With respect to FIG. 7d a further example of an example of an ‘open’audio sub-menu user interface representation is shown whereby bothiconic representations and labels are used to provide information andoptions to the user. For example the iconic or pictorial representationsshown in FIG. 7b are used to represent the type of the audio source,internal or external, and their coupling to the apparatus with anumerical label 633, 635 identifying each of the type. Also the labelrepresentations of the options associated with the audio sources ‘NOISECANCELLATION’ 629 and ‘AUTOMATIC SOUND CONTROL’ 639 representinginformation on the use or processing of the audio sources.

In some embodiments the parameter/signal visualizer 105 can in someembodiments comprise an advanced audio control user interface visualizer255. The advanced audio control user interface visualizer 255 can insome embodiments be configured to generate visual representations forenabling control of audio sources (for example an internal or externalmicrophone).

With respect to FIGS. 8a to 8j examples of visual representationsgenerated by the advanced audio control user interface visualizer 255are shown.

The advanced audio control user interface visualizer 255 can beconfigured for example as shown in FIG. 8a to generate a first advancedaudio control UI representation 701 on the display. The first advancedaudio control Ul representation 701 can be in some embodiments arepresentation of a microphone in an arrow shaped window with the arrowpointing inwards to indicate that by selecting this representationadvanced audio UI controls can be displayed. The advanced audio controlUI representation 701 can be considered to be a ‘closed’ representationwhich is configured to ‘open’ when selected or interfaced with. It wouldbe understood that in other embodiments the activation of the advancedaudio UI controls could be implemented without a specific advanced audiocontrol UI visual representation change, or could be a change other thanthat described above such as colour, shading, or effect. Furthermore itwould be understood that any transition described herein and representedby a visual representation change could be represented by an audiorepresentation, such as an audio clip or sound being output and/or ahaptic representation of a change such as a physical movement of theapparatus or display.

In a manner as described above the first advanced audio control Ulrepresentation 701 can in some embodiments be interfaced with byselecting the representation, for example by touching the display withina region defined by the representation. The UI input 113 can then insuch embodiments pass the touch selection to the UI input processor 107.In some embodiments as well as the U I input processor 107 passing thecontrol signal to the audio source controller 101 the UI input processor107 is configured to pass a control signal to the parameter/signalvisualizer 105. In such embodiments the control signal can be used bythe parameter/signal visualizer to change the representation passed tothe display. Thus for example in the advanced audio control UIrepresentation example by selecting or touching the advanced audiocontrol UI representation 701, the UI input 113 passes the ‘touch’ inputto the U I input processor 107 which provides a control signal to theadvanced audio control user interface visualizer 255 to open up theadvanced UI audio control visual representation (or if open to close theadvanced UI audio control visual representation).

With respect to FIG. 8b an example of an ‘open’ advanced UI audiocontrol visual representation is shown. The ‘open’ advanced UI audiocontrol visual representation in this example comprises an ‘open’advanced UI audio control visual representation 711, shown as amicrophone in an arrow similar to that shown in the ‘closed’ advanced UIaudio control visual representation but with the direction of the arrowreversed to point to the right rather than the left and further ‘open’or information windows within which information and control of thevarious audio sources can be implemented.

The examples shown with respect to the advanced UI audio control visualrepresentations in FIGS. 8b to 8j show each audio source as a window 713located within which are indicators associated with each source. In someembodiments advanced audio control user interface visualizer 255generates a visual representation representing the type of audio source(for example an external microphone audio source shown by the pictorialrepresentation of a microphone 715) and furthermore the characteristicsof the audio source (for example the Bluetooth representation 717).

Furthermore as shown in FIGS. 8b and 8c , the advanced audio controluser interface visualizer 255 can in some embodiments generate visualrepresentations to display and/or enable control of information withregards to the processing or control of the audio signals. For examplethe advanced audio control user interface visualizer 255 can in someembodiments generate ‘NOISE CANCELLATION’ 629 and ‘AUTOMATIC SOUNDCONTROL’ 639 labels with associated toggle, tick or selection boxes 721and 725 representing information on the use or processing of the audiosources.

In some embodiments the advanced audio control user interface visualizer255 can by the colour or shading of the box 713 indicate the gain beingapplied to the audio signal received from the audio source representedby the visual representation. Thus in some embodiments a green colourindicates a 100% input gain, an amber colour a 50%-99% input gain and ared input a 0%-49% input gain. In some embodiments such as shown in FIG.8c the advanced audio control user interface visualizer 255 indicate anactivation or deactivation of the audio source by a ‘bar’ visualrepresentation 741. As described herein by interacting with the audiosource box 713, such as by touching the visual representation of theaudio source box on the display, the UI input determines the touch andthe UI input processor 107 on receiving the detected touch of the audiosource box 713 can be configured to control both the toggling of theaudio source by controlling the switch module 104 and/or audio sourcecontroller 101 and also controlling the parameter/signal visualizer toapply or to remove the ‘bar’ visual representation. It would beunderstood that shading and/or colour schemes could further beimplemented to indicate the operational status of the audio source, forexample changing the colour of the audio source box visualrepresentation from green to red on selecting the audio source to be notreceived or processed.

In some embodiments the advanced audio control user interface visualizer255 can be configured to generate a visual representation reflecting therelative proportions of two audio sources used in mixing by generating afirst audio source box 713 and a second audio source box 751 linked by avisual representation of a track or rod 753 on which a slider visualrepresentation 755 indicates of the relative gain applied in the mixermodule 106 mixing the audio signals from each of the sources. Thus forexample in FIG. 8d the slider is substantially indicative that the audiosource input is that received from the external Bluetooth microphone asthe slider is a substantially closer to the Bluetooth microphone orfirst audio source box visual representation 713 than the second orinternal microphone audio source box visual representation 751.

It would be understood that in such embodiments the mixing can bechanged by interfacing with the slider 755, for example touching andsliding a finger along the visual representation of the track or rod753. Furthermore it would be appreciated that in some embodiments theadvanced audio control user interface visualizer 255 can implementcolour or shading schemes to further indicate to the user the currentproportions of the audio signal being applied to the mix.

In some embodiments the advanced audio control user interface visualizer255 can be configured to generate visual representations whereby eachaudio source box is located on a track or rod providing control of eachaudio source audio signal. For example as shown in FIG. 8e each audiosource box 713, 751 is a virtual slider on a respective track or rod763, 761 the position along the rod providing or representing a controlsignal for mixing the detected and displayed audio signals.

Thus the user can ‘fade’ both audio sources to zero by interfacing witheach audio source box, in other words moving the audio source box to thebottom of the track or rod as shown in FIG. 8f . Furthermore as theinput of the audio source signal reaches zero the advanced audio controluser interface visualizer 255 can generate visual representation bars741, 771 to be applied to the audio source boxes 713, 751.

In some embodiments more than one interaction or interfacing mode can beapplied to each audio source. As described herein by ‘moving’ the audiosource box on the track or rod the gain or relative gain can of theaudio signal associated with each audio source can be controlled.However in some embodiments of the application by interacting but notmoving the audio source box a further operation can be carried out. Insuch embodiments the further operation can be to display characteristicsabout the audio signal. With respect to FIG. 8g such an example isshown. For example in some embodiments by selecting the audio source boxa sub-window 781 can be generated by the advanced audio control userinterface visualizer 255 within which can be displayed a time domainrepresentation 783 of the audio signal associated with the audio sourceand a label ‘30 sec’ 785 indicating the time span of the time domainrepresentation. In some further embodiments the advanced audio controluser interface visualizer 255 can in dependent of the user interfacecontrol processor signal generate a frequency domain representation. Insome embodiments the interaction with the audio source box furthercontrols the period of the domain representation, so that the longer thebox is ‘touched’ the longer the period displayed.

In some embodiments the advanced audio control user interface visualizer255 can generate visual representations for more than two audio sources.With respect to

FIG. 8h a first example is shown which is similar to the examples shownin FIGS. 8e to 8g where each audio source has its own rod or track onwhich a visual representation of the audio source, the audio sourceboxes 713, 714, 751 are sliders controlling the gain applied at themixer module 106 to the associated audio signals. With respect to FIG.8i a further example is shown which is similar to the example shown inFIG. 8d where each audio source is represented by an audio box and theaudio boxes are coupled together by a series of visual representation ofrods with a central point 794 representing the control signal where allsignals are mixed equally. The control of the gain of applied at themixer module 106 in such examples can be implemented by a virtual slider795 operating between the audio source boxes.

With respect to FIG. 8j another visual representation example is shownwhereby the audio source boxes 713, 714, and 751 are each associatedwith a rod or track 761, 762, 763 and on each rod or trackrepresentation is a slider 781, 783 785 configured to be interacted orinterfaced with. Thus for example in such embodiments themixing/filtering control can be configured to be dependent on theinteraction of the position of the slider on the track and informationon the audio source control can be configured to be dependent oninteraction with the audio box. Although the example with respect toFIG. 8j shows three audio sources it would be understood that anysuitable number of audio sources can be represented.

In some embodiments the advanced audio control user interface visualizer255 can generate the visual representations such that they overlie otherimages. For example in some embodiments the UI 15 is configured todisplay an image taken using a camera, the advanced audio control userinterface visualizer 255 is configured based on the detected audiosource information provided from the sensors and processed by the audiosource controller audio source determiner 102 associate each detectedaudio source with a position in the image taken from the camera. Thusfor example as shown in FIG. 9, the image of a concert taken by thecamera is overlaid with a first audio source box 801 representing theapparatus or devices internal microphone, a second audio source box 803representing the external microphone of the vocalist, and a third audiosource box 805 representing the external microphone of a furtherapparatus or device located nearer the stage than the apparatus. In sucha manner it would be possible to select or mix between the audio signalsreceived from each audio source to capture a better audio signal.

In other words in some embodiments there can comprise means fordisplaying the visual representation dependent on the position of the atleast one audio source.

Further examples of the implementation of embodiments of the applicationcan be seen in FIGS. 4 and 5 a/5 b. In the example shown in FIG. 4 aninterviewer 201 can be equipped with suitable audio/video recordingapparatus 10 according to embodiments of the application. Theinterviewer 201 can furthermore be equipped with a Bluetooth device withmicrophones 31 a and 31 b which can be detected by the apparatus audiosource determiner and controlled with respect to the visualrepresentations generated by the parameter/signal visualizer such thatmixing the audio sources enables the interviewer 201 to interview theinterviewees 203, 205.

In the examples shown in FIGS. 5a and 5b a studio style operation can beshown where the apparatus 10 is located at position 1 305 in FIG. 5a andin position 2 in FIG. 5b and is configured to receive audio signals fromexternal microphones 31 a and 31 b at microphone position 1 307 andposition 2 309 respectively. In both examples the microphone signals canbe monitored and mixed directed on the apparatus 10 thus overcoming theneed for sophisticated and expensive mixing equipment.

Although the above examples describe embodiments of the inventionoperating within an electronic device 10 or apparatus, it would beappreciated that the invention as described below may be implemented aspart of any audio processor.

Thus, for example, embodiments of the invention may be implemented in anaudio processor which may implement audio processing over fixed or wiredcommunication paths.

Thus user equipment may comprise an audio processor such as thosedescribed in embodiments of the invention above.

It shall be appreciated that the term electronic device and userequipment is intended to cover any suitable type of wireless userequipment, such as mobile telephones, portable data processing devicesor portable web browsers.

In general, the various embodiments of the invention may be implementedin hardware or special purpose circuits, software, logic or anycombination thereof. For example, some aspects may be implemented inhardware, while other aspects may be implemented in firmware or softwarewhich may be executed by a controller, microprocessor or other computingdevice, although the invention is not limited thereto. While variousaspects of the invention may be illustrated and described as blockdiagrams, flow charts, or using some other pictorial representation, itis well understood that these blocks, apparatus, systems, techniques ormethods described herein may be implemented in, as non-limitingexamples, hardware, software, firmware, special purpose circuits orlogic, general purpose hardware or controller or other computingdevices, or some combination thereof.

Therefore in summary there is in at least one embodiment an apparatuscomprising: an audio source determiner configured to determine at leastone audio source; a visualizer configured to generate a visualrepresentation associated with the at least one audio source; and acontroller configured to process an audio signal associated with the atleast one audio source dependent on interaction with the visualrepresentation.

The embodiments of this invention may be implemented by computersoftware executable by a data processor of the mobile device, such as inthe processor entity, or by hardware, or by a combination of softwareand hardware. Further in this regard it should be noted that any blocksof the logic flow as in the Figures may represent program steps, orinterconnected logic circuits, blocks and functions, or a combination ofprogram steps and logic circuits, blocks and functions. The software maybe stored on such physical media as memory chips, or memory blocksimplemented within the processor, magnetic media such as hard disk orfloppy disks, and optical media such as for example DVD and the datavariants thereof, CD.

Thus at least one embodiment comprises a computer-readable mediumencoded with instructions that, when executed by a computer perform:determining at least one audio source; generating a visualrepresentation associated with the at least one audio source; andprocessing an audio signal associated with the at least one audio sourcedependent on interaction with the visual representation.

The memory may be of any type suitable to the local technicalenvironment and may be implemented using any suitable data storagetechnology, such as semiconductor-based memory devices, magnetic memorydevices and systems, optical memory devices and systems, fixed memoryand removable memory. The data processors may be of any type suitable tothe local technical environment, and may include one or more of generalpurpose computers, special purpose computers, microprocessors, digitalsignal processors (DSPs), application specific integrated circuits(ASIC), gate level circuits and processors based on multi-core processorarchitecture, as non-limiting examples.

Embodiments of the inventions may be practiced in various componentssuch as integrated circuit modules. The design of integrated circuits isby and large a highly automated process. Complex and powerful softwaretools are available for converting a logic level design into asemiconductor circuit design ready to be etched and formed on asemiconductor substrate,

Programs, such as those provided by Synopsys, Inc. of Mountain View,Calif. and Cadence Design, of San Jose, Calif. automatically routeconductors and locate components on a semiconductor chip using wellestablished rules of design as well as libraries of pre-stored designmodules. Once the design for a semiconductor circuit has been completed,the resultant design, in a standardized electronic format (e.g., Opus,GDSII, or the like) may be transmitted to a semiconductor fabricationfacility or “fab” for fabrication.

As used in this application, the term ‘circuitry’ refers to all of thefollowing:

-   -   (a) hardware-only circuit implementations (such as        implementations in only analog and/or digital circuitry) and    -   (b) to combinations of circuits and software (and/or firmware),        such as: (i) to a combination of processor(s) or (ii) to        portions of processor(s)/software (including digital signal        processor(s)), software, and memory(ies) that work together to        cause an apparatus, such as a mobile phone or server, to perform        various functions and    -   (c) to circuits, such as a microprocessor(s) or a portion of a        microprocessor(s), that require software or firmware for        operation, even if the software or firmware is not physically        present.

This definition of ‘circuitry’ applies to all uses of this term in thisapplication, including any claims. As a further example, as used in thisapplication, the term ‘circuitry’ would also cover an implementation ofmerely a processor (or multiple processors) or portion of a processorand its (or their) accompanying software and/or firmware. The term‘circuitry’ would also cover, for example and if applicable to theparticular claim element, a baseband integrated circuit or applicationsprocessor integrated circuit for a mobile phone or similar integratedcircuit in server, a cellular network device, or other network device.

The foregoing description has provided by way of exemplary andnon-limiting examples a full and informative description of theexemplary embodiment of this invention. However, various modificationsand adaptations may become apparent to those skilled in the relevantarts in view of the foregoing description, when read in conjunction withthe accompanying drawings and the appended claims. However, all such andsimilar modifications of the teachings of this invention will still fallwithin the scope of this invention as defined in the appended claims.

1-20. (canceled)
 21. An apparatus comprising: at least one processor;and at least one non-transitory memory including computer program code;the at least one memory and the computer program code configured to,with the at least one processor, cause the apparatus at least toperform: receive a plurality of audio signals from a plurality ofmicrophones; provide at least one visual representation with a userinterface, wherein the user interface is configured to receive at leastone interaction, wherein the at least one visual representation isconfigured to indicate, at least, a first microphone usage for a firstmicrophone of the plurality of microphones and a second microphone usagefor a second microphone of the plurality of microphones; and mix, atleast, the plurality of audio signals based, at least partially, on theat least one received interaction, wherein the at least one receivedinteraction comprises an interaction relative to the at least one visualrepresentation, wherein the at least one received interaction isconfigured to control the mixing of the plurality of received audiosignals.
 22. The apparatus of claim 21, wherein the at least onereceived interaction is configured to control at least one of: ananalogue gain, or a digital gain, applied to at least one of: a firstaudio signal associated with the first microphone, or a second audiosignal associated with the second microphone.
 23. The apparatus of claim21, wherein the at least one received interaction is configured tocause: a first gain to be applied to a first audio signal of theplurality of audio signals associated with the first microphone, and asecond gain to be applied to a second audio signal of the plurality ofaudio signals associated with the second microphone, wherein the secondgain is smaller than the first gain.
 24. The apparatus of claim 21,wherein the at least one visual representation comprises: a first visualrepresentation related to the first microphone, and a second visualrepresentation related to the second microphone.
 25. The apparatus ofclaim 24, wherein the at least one visual representation comprises: avisual representation of a slider between the first visualrepresentation and the second visual representation, wherein the mixingof the plurality of audio signals is based on a position of the visualrepresentation of the slider between the first visual representation andthe second visual representation.
 26. The apparatus of claim 25, whereinthe at least one received interaction is configured to move the visualrepresentation of the slider relative to the first visual representationand the second visual representation.
 27. The apparatus of claim 24,wherein the at least one visual representation comprises: a third visualrepresentation related to a third microphone of the plurality ofmicrophones, and at least one visual representation related to a gain,wherein the first visual representation, the second visualrepresentation, and the third visual representation are in anarrangement configured to visually indicate that a first audio signalassociated with the first microphone, a second audio signal associatedwith the second microphone, and a third audio signal associated with thethird microphone are to be mixed equally, wherein the at least onevisual representation related to the gain is configured to indicate thegain applicable to the first audio signal, the second audio signal, andthe third audio signal, wherein the at least one received interactioncomprises an interaction with the at least one visual representationrelated to the gain and is configured to modify the gain.
 28. Theapparatus of claim 21, wherein at least one visual representation isconfigured to indicate a proportion of the plurality of audio signalsused in the mixing.
 29. The apparatus of claim 28, wherein theproportion of the plurality of audio signals is indicated via at leastone of: a color scheme, or a shading scheme.
 30. The apparatus of claim21, wherein the at least one received interaction is configured to causeat least one of: the first microphone usage, or the second microphoneusage to be zero for the mixing.
 31. The apparatus of claim 21, whereinthe at least one received interaction is configured to control at leastone of: time sample mixing, or frequency domain mixing of the pluralityof audio signals.
 32. The apparatus of claim 21, wherein the pluralityof microphones comprises at least one of: at least one microphone thatis physically part of the apparatus and at least one microphone that isphysically separate from the apparatus, or at least two microphones thatare physically part of the apparatus.
 33. The apparatus of claim 21,wherein the at least one interaction is configured to control at leastone of: the first microphone usage, or the second microphone usage. 34.A method comprising: receiving, with an apparatus, a plurality of audiosignals from a plurality of microphones; providing at least one visualrepresentation with a user interface, wherein the user interface isconfigured to receive at least one interaction, wherein the at least onevisual representation is configured to indicate, at least, a firstmicrophone usage for a first microphone of the plurality of microphonesand a second microphone usage for a second microphone of the pluralityof microphones; and mixing, at least, the plurality of audio signalsbased, at least partially, on the at least one received interaction,wherein the at least one received interaction comprises an interactionrelative to the at least one visual representation, wherein the at leastone received interaction is configured to control the mixing of theplurality of received audio signals.
 35. The method of claim 34, whereinthe at least one received interaction is configured to cause: a firstgain to be applied to a first audio signal of the plurality of audiosignals associated with the first microphone, and a second gain to beapplied to a second audio signal of the plurality of audio signalsassociated with the second microphone, wherein the second gain issmaller than the first gain.
 36. The method of claim 34, wherein the atleast one visual representation comprises: a first visual representationrelated to the first microphone, a second visual representation relatedto the second microphone, and a visual representation of a sliderbetween the first visual representation and the second visualrepresentation, wherein the mixing of the plurality of audio signals isbased on a position of the visual representation of the slider betweenthe first visual representation and the second visual representation.37. The method of claim 34, wherein the at least one receivedinteraction is configured to cause at least one of: the first microphoneusage, or the second microphone usage to be zero for the mixing.
 38. Themethod of claim 34, wherein the plurality of microphones comprises atleast one of: at least one microphone that is physically part of theapparatus and at least one microphone that is physically separate fromthe apparatus, or at least two microphones that are physically part ofthe apparatus.
 39. The method of claim 34, wherein the at least oneinteraction is configured to control at least one of: the firstmicrophone usage, or the second microphone usage.
 40. A non-transitorycomputer-readable medium comprising program instructions stored thereonwhich, when executed with at least one processor, cause the at least oneprocessor to: receive a plurality of audio signals from a plurality ofmicrophones; provide at least one visual representation with a userinterface, wherein the user interface is configured to receive at leastone interaction, wherein the at least one visual representation isconfigured to indicate, at least, a first microphone usage for a firstmicrophone of the plurality of microphones and a second microphone usagefor a second microphone of the plurality of microphones; and mix, atleast, the plurality of audio signals based, at least partially, on theat least one received interaction, wherein the at least one receivedinteraction comprises an interaction relative to the at least one visualrepresentation, wherein the at least one received interaction isconfigured to control the mixing of the plurality of received audiosignals.